Reputation serves as a powerful mechanism for promoting cooperation in multi-agent systems, as agents are more inclined to cooperate with those of good social standing. While existing multi-agent reinforcement learning methods typically rely on predefined social norms to assign reputations, the question of how a population reaches a consensus on judgement when agents hold private, independent views remains unresolved. In this paper, we propose a novel bottom-up reputation learning method, Learning with Reputation Reward (LR2), designed to promote cooperative behaviour through rewards shaping based on assigned reputation. Our agent architecture includes a dilemma policy that determines cooperation by considering the impact on neighbours, and an evaluation policy that assigns reputations to affect the actions of neighbours while optimizing self-objectives. It operates using local observations and interaction-based rewards, without relying on centralized modules or predefined norms. Our findings demonstrate the effectiveness and adaptability of LR2 across various spatial social dilemma scenarios. Interestingly, we find that LR2 stabilizes and enhances cooperation not only with reward reshaping from bottom-up reputation but also by fostering strategy clustering in structured populations, thereby creating environments conducive to sustained cooperation.

Consider the scenario where an infinite number of players (i.e., the \textit{thermodynamic} limit) find themselves in a Prisoner's dilemma type situation, in a \textit{repeated} setting. Is it reasonable to anticipate that, in these circumstances, cooperation will emerge? This paper addresses this question by examining the emergence of cooperative behaviour, in the presence of \textit{noise} (or, under \textit{selection pressure}), in repeated Prisoner's Dilemma games, involving strategies such as \textit{Tit-for-Tat}, \textit{Always Defect}, \textit{GRIM}, \textit{Win-Stay, Lose-Shift}, and others. To analyze these games, we employ a numerical Agent-Based Model (ABM) and compare it with the analytical Nash Equilibrium Mapping (NEM) technique, both based on the \textit{1D}-Ising chain. We use \textit{game magnetization} as an indicator of cooperative behaviour. A significant finding is that for some repeated games, a discontinuity in the game magnetization indicates a \textit{first}-order \textit{selection pressure/noise}-driven phase transition. The phase transition is particular to strategies where players do not severely punish a single defection. We also observe that in these particular cases, the phase transition critically depends on the number of \textit{rounds} the game is played in the thermodynamic limit. For all five games, we find that both ABM and NEM, in conjunction with game magnetization, provide crucial inputs on how cooperative behaviour can emerge in an infinite-player repeated Prisoner's dilemma game.

This work extends an existing virtual multi-agent platform called RoboSumo to create TripleSumo -- a platform for investigating multi-agent cooperative behaviors in continuous action spaces, with physical contact in an adversarial environment. In this paper we investigate a scenario in which two agents, namely'Bug' and'Ant', must team up and push another agent'Spider' out of the arena. To tackle this goal, the newly added agent'Bug' is trained during an ongoing match between'Ant' and'Spider'. 'Bug' must develop awareness of the other agents' actions, infer the strategy of both sides, and eventually learn an action policy to cooperate. The reinforcement learning algorithm Deep Deterministic Policy Gradient (DDPG) is implemented with a hybrid reward structure combining dense and sparse rewards. The cooperative behavior is quantitatively evaluated by the mean probability of winning the match and mean number of steps needed to win.

Guilt aversion induces experience of a utility loss in people if they believe they have disappointed others, and this promotes cooperative behaviour in human. In psychological game theory, guilt aversion necessitates modelling of agents that have theory about what other agents think, also known as Theory of Mind (ToM). We aim to build a new kind of affective reinforcement learning agents, called Theory of Mind Agents with Guilt Aversion (ToMAGA), which are equipped with an ability to think about the wellbeing of others instead of just self-interest. To validate the agent design, we use a general-sum game known as Stag Hunt as a test bed. As standard reinforcement learning agents could learn suboptimal policies in social dilemmas like Stag Hunt, we propose to use belief-based guilt aversion as a reward shaping mechanism. We show that our belief-based guilt averse agents can efficiently learn cooperative behaviours in Stag Hunt Games.

The connected automated vehicle has been often touted as a technology that will become pervasive in society in the near future. One can view an automated vehicle as having Artificial Intelligence (AI) capabilities, being able to self-drive, sense its surroundings, recognise objects in its vicinity, and perform reasoning and decision-making. Rather than being stand alone, we examine the need for automated vehicles to cooperate and interact within their socio-cyber-physical environments, including the problems cooperation will solve, but also the issues and challenges. We review current work in cooperation for automated vehicles, based on selected examples from the literature. We conclude noting the need for the ability to behave cooperatively as a form of social-AI capability for automated vehicles, beyond sensing the immediate environment and beyond the underlying networking technology.

Cooperation is one of the hallmarks of being human. We are extremely social compared to other species. On a regular basis, we all enter into helping others in small but important ways, whether it be letting someone out in traffic or giving a tip for good service. We do this without any guarantee of payback. Donations are made at a small personal cost but with a bigger benefit to the recipient. This form of cooperation, or donation to others, is called indirect reciprocity and helps human society to thrive.

Cooperation is one of the hallmarks of being human. We are extremely social compared to other species. On a regular basis, we all enter into helping others in small but important ways, whether it be letting someone out in traffic or giving a tip for good service. We do this without any guarantee of payback. Donations are made at a small personal cost but with a bigger benefit to the recipient. This form of cooperation, or donation to others, is called indirect reciprocity and helps human society to thrive.

Cooperation is one of the hallmarks of being human. We are extremely social compared to other species. On a regular basis, we all enter into helping others in small but important ways, whether it be letting someone out in traffic or giving a tip for good service. We do this without any guarantee of payback. Donations are made at a small personal cost but with a bigger benefit to the recipient. This form of cooperation, or donation to others, is called indirect reciprocity and helps human society to thrive.

Cooperation is one of the hallmarks of being human. We are extremely social compared to other species. On a regular basis, we all enter into helping others in small but important ways, whether it be letting someone out in traffic or giving a tip for good service. We do this without any guarantee of payback. Donations are made at a small personal cost but with a bigger benefit to the recipient.